|
Author |
Thread Statistics | Show CCP posts - 9 post(s) |
|
CCP Rattati
C C P C C P Alliance
9555
|
Posted - 2014.10.29 07:22:00 -
[1] - Quote
Dear data fanatics,
While I am currently writing a bigger piece on data and meta, I am going to run a little test case by the forums, both to battle-test the visualizations and to get some initial feedback.
I will try to explain the thinking and supporting data as clearly as possible, even though this would be more fun as a presentation in front of a live audience.
Case: The Rail Rifle
KPI's: Kills, Spawns and K/S Sample: Prototype Rifles in Public Contracts since May 2014 (post 1.8 through all hotfixes) Data Insights: The Rail Rifle has the highest K/S in the sample, gets the most kills and is the most used Rifle currently Main Theory: The Rail Rifle is too effective Forum Support: Yes Secondary Theory: The following Rifles are not effective enough, Assault Scrambler, Burst Assault, Tactical Assault, and Assault Rail. Forum Support: Yes
1)Supporting Data GÇô K/S Chart
The White Line is the average for all Kills and Spawns of the Sample. The Black Line is the 7 day K/S of all Rifles. Those two lines establish a baseline for Rifles versus other weapons, and a baseline for individual Rifles against the Rifle average.
In the chart we can see that the previous changes we have made are reflected in the data, Scrambler Rifle nerf in Delta shows the K/S quickly declining and rallying, same for the Combat Rifle, multiple Tactical Assault Rifle buffs are reflected in the K/S growth over time and the Breach AR can be consider competitive and right on the White Line.
We can also see that the Spread (Highest - Lowest) is getting smaller, from 1.85-0.55 = 1.3 in May to 1.70-1.0 = 0.7, almost a 50% decrease. That tells us that players that use these Rifles are able to get comparative performances.
Now notice that the White Line, the average, is not in the middle of the lines as one might expect with balance getting better. This is because there are more players killing and spawning with the Rifles above the White Line, than the Rifles below the line. This is a cause of concern and prompts the second Chart.
2)Supporting Data GÇô Kills Chart
Looking at this chart, there are quite a few observations to make.
The sum of the top three Rifles is more than 50%, with ten Rifle options, there should be an equal amount above and below 50%. The dominance of the Rail Rifle is the reason for this situation, actually, the 5 next Rifles are fairly well balanced with regard to Kills. Then we have the (perpetual) underperformers, the Assault Scrambler, Burst Assault, Tactical and Assault Rail.
The Assault Scrambler was buffed in the balance hotfix 2013.10 along with the Assault Rail Rifle. Both had been on a downward trend for some time. It is quite possible that the Burst Assault has not been discovered as it is not a high profile weapon, anecdotally and stats wise it should be competitive so a further buff was deemed inappropriate.
Since, historically, the Combat Rifle and Scrambler Rifle both broke the 2.0 K/S line, they were nerfed as a responese but the Rail Rifle was under the radar for a while, especially with the downward trend in K/S after Delta, as it was not obviously overpowered, until combined with the Kills data.
It is always possible that a weapon has a lot of Kills, e.g., familiarity, prevalence in Starter Fits, time and SP to spec into another weapon, etc, while not being overpowered. ThatGÇÖs why only one KPI should never be used on its own to balance.
3)Supporting Data GÇô Performance Charts
Comparing these three historically powerful Rifles gives further insights, and especially when the forums repeatedly call for GÇ£the Combat RifleGÇ¥ to be next. In fact, it has already been nerfed in Delta.
Since Spawns and Kills depend on the daily activity, all these numbers have been normalized by dividing by the total daily Spawns and Kills, respectively.
The Green Area are the Spawns, reduction in these indicate the lack of interest by the playerbase, maybe because of stats, handling, migration to another weapon or bad synergy between dropsuit and weapon, or something else.
The Green Line is the average Spawns, when the Green Area dips above or below that line, something is likely causing that change in trends.
Similarly, the White Bars are the Kills and the White Line the average Kills. Again, being above or below a historical average usually indicates behavior trend changes.
Finally, the Red Line is the 7 day Kills per Spawn, and the Red Dotted Line the average K/S.
3a) Supporting Data GÇô Combat Rifle
3b) Supporting Data GÇô Scrambler Rifle
3c) Supporting Data GÇô Rail Rifle
The design of the Rail Rifle is to be a highly accurate, long range weapon. Following that design, and taking into account the massive, and continuous growth in Kills, the Rail RifleGÇÖs Close Combat ability was reduced severely by increasing Kick and Charge-up Time, with no change to ADS Dispersion or Kick.
We will monitor these changes and player behavior, and make sure that the Rail Rifle does not become non-viable. The optimal situation would be that Rail Rifle users migrate to lesser used Rifles. If they mass-migrate to a single rifle, we will need to take some action in the near future.
Please comment and give feedback
"As well as stupid, Rattati is incredibly slow and accident-prone, and cannot even swim"
|
|
|
CCP Rattati
C C P C C P Alliance
9566
|
Posted - 2014.10.29 09:54:00 -
[2] - Quote
Tesfa Alem wrote:This is a bit of an eye opener and thanks for sharing.
However, it still looks like a crucial element is missing, how rifles are being used in game. In my humble opinion, there is no reason for all ten rifles to be getting equal amounts of kills per spawn.
Do you take into conideration for example usage and design philosophy first and see if the gun is overperforming or tweak the numbers to follow design philosphy afterwards?
Using the Rail Rifle: Design to be a long range in line with caldari lore. Is there data to show that wow, the RR is getting alot of close range kills this is unusual for a longe range weapon? Or rather the RR is getting alot of kills everywhere lets nerf the CQC because its long range weapon?
Before any proposal i'm sure you do need the stats to back you up. I'm curious as too what stats unrelated to kills are taken into consideration, also other factors such as ease of use, fitting costs, market purchases, damage output etc. By this reasoning its easy to see why shotguns remain untouched, a scout might one shot a medium frameor two/ three shot a heavy and get killed right afterwards by another scout, it might look just fine according to these metrics bu IMO unbalanced in terms of gameplay.
It doesn't really matter though, no single rifle should be doing 50% of all Rifle kills. We could also just have nerfed damage until it was not as extremely good at all ranges, hurting the weapons primary function.
"As well as stupid, Rattati is incredibly slow and accident-prone, and cannot even swim"
|
|
|
CCP Rattati
C C P C C P Alliance
9610
|
Posted - 2014.10.29 16:23:00 -
[3] - Quote
MINA Longstrike wrote:So what I'm seeing here, is that the rail rifle was roughly balanced and you decided to hit it with a harsh nerf anyways, because you didn't like the fact that it was being used in a game where the map design accommodates two things - long range poking or short-ish range brawling.
Range is king in shooters and always has been unless map design puts you in knife-fights all the time.
:narrows eyes: ...not sure if serious
"As well as stupid, Rattati is incredibly slow and accident-prone, and cannot even swim"
|
|
|
CCP Rattati
C C P C C P Alliance
9613
|
Posted - 2014.10.29 16:28:00 -
[4] - Quote
MINA Longstrike wrote:Because it doesn't seem like the data you have supports the nerf that was made.
I believe it does, and have made a detailed fact based argument to support it.
"As well as stupid, Rattati is incredibly slow and accident-prone, and cannot even swim"
|
|
|
CCP Rattati
C C P C C P Alliance
9613
|
Posted - 2014.10.29 16:32:00 -
[5] - Quote
MINA Longstrike wrote:^Completely serious. There hasn't been any data provided that suggests that the RR was overperforming in CQC or that it was even getting a much much higher KDR than other weapons. Just a lot of data that says "people like using it".
lol, if it wasn't being used in CQC, why does an exclusive CQC nerf bother you, ADS wasn't touched.
"As well as stupid, Rattati is incredibly slow and accident-prone, and cannot even swim"
|
|
|
CCP Rattati
C C P C C P Alliance
9644
|
Posted - 2014.10.29 17:16:00 -
[6] - Quote
MINA Longstrike wrote:CCP Rattati wrote:MINA Longstrike wrote:^Completely serious. There hasn't been any data provided that suggests that the RR was overperforming in CQC or that it was even getting a much much higher KDR than other weapons. Just a lot of data that says "people like using it". lol, if it wasn't being used in CQC, why does an exclusive CQC nerf bother you, ADS wasn't touched. Because it paints it into a corner like the laser rifle, you are turning a racial servicemans weapon (a weapon that should be functional across all ranges) into a specialist weapon (like the laser or massdriver or PLC). You are dramatically and arbitrarily limiting a weapons ability to play the objective in skirmish or domination style matches, simply because it is 'used' more frequently than you think it should be. I'd still like to see data provided for KDA's at various ranges, if you actually have such information. I'll point you towards Jasyn Larrisen's posts because he's better at expressing the underpinning issues of this. There are so many better ways that you could have done this Rattati. I'm not saying that the weapon didn't need to be tweaked or that I want it completely undone - I'm saying that what was done is overdone.
Just like the AR is painted into a corner by having very short range?
Arbitrarily? It means "based solely on one's opinion", does this seem a very arbitrary process?
"Used" more? No, it kills more. The Chart shows Kills, not Spawns.
"As well as stupid, Rattati is incredibly slow and accident-prone, and cannot even swim"
|
|
|
CCP Rattati
C C P C C P Alliance
9719
|
Posted - 2014.10.29 23:17:00 -
[7] - Quote
Alena Ventrallis wrote:Let me clarify some things about my opinions.
When the RR received the 4x increase to hipfire kick, the ARR also received a 4x increase to hipfire kick. This is what my issue is. If this was an accident I'd like it fixed posthaste. If it was intentional, I'd like to know why so I can understand the reasoning better and can formulate a proper opinion on it.
We will look into that post haste
"As well as stupid, Rattati is incredibly slow and accident-prone, and cannot even swim"
|
|
|
CCP Rattati
C C P C C P Alliance
9852
|
Posted - 2014.10.31 14:20:00 -
[8] - Quote
Rynx Sinfar wrote:Thanks for posting this and as someone who has not logged into the game in quite some time I am very eager to see more. That said I'm gonna make fun of you a bit. First, for the Kills chart... come on man, that axis. Second, for the K/S chart you may as well be making the lines vibrate with those colors. You have 12 lines and use 4 shades of orange/yellow, 4 shades of blue, 3 shades of black/white, and 1 green. I have to zoom in and constantly check the legend because those blues are so damn similar. The blues are also the ones with the most other lines around them. Maybe distribute those colors a bit better? Also maybe some grouping? I don't know what it is with you guys but you put things in the most random orders. Grouping colors to types (along with better visibility) would make it much easier to find what I'm looking for. If all the ARs are green I can easily see overall AR performance as well. I'm not saying you need to spend a ton of time on this but maybe have some sort of pattern? I'll skip style critiques for the other charts as I think you get the idea. Also I know you guys mostly use Google Docs for distributing data but maybe you could try... Google Docs? You may be able to find some successful examples somewhere. It removes a lot of the color variation concern when i can just highlight lines. Also I love the inclusion of events, however it is hard to determine where the actual "line" is for them. For example Bravo released 10 days before the Expert challenge event, but it is hard to see that fact on the chart. Which leads me to my next point that you don't need to do all this personally. I just spent a ton of my time working on something based on your data. Clearly you have a community willing to foolishly waste their own time. I've logged in maybe 1 to 2 times in the past half year and I was willing to throw in time. Give us the raw data so anyone digging into the information can have an easier time. I mean the numbers are technically there. I could overlay a grid and get decent estimates pretty quickly. But why make me work for it? Plus it makes things easier with google docs. That said definitely give explanations because yes people are going to fit data to match their own conclusions rather than search for all possible conclusions. I caught myself doing that already. I'm no statistician. I am not even 100% sure if I read the data from the K/S sheet right because of those freaking blues. As for the data itself, I know ranges were normalized in Bravo so my guess is pre-Bravo Advanced weapon data is way different, but this guy would imply that a lot of action is going on in non-protoland. Even if the data is only Bravo onwards I'd be curious to have a look. I know historically many people who could run proto would not (especially in public matches). I imagine you've looked at the data yourself but I can't help feeling like the representative group for this data can only be a smaller percent of the player population (even if only compared to advanced gear use). I know in many ways giving us unfiltered data can lead to simple inaccurate statements that start angry mobs (Subdredditors do go on Reddit now and then after all) but then what's the worst that might happen? People getting into a huff over absurd stuff they don't properly understand and complaining on the forums? Oh noooooo.... P.S. Also those yellows. I'm pretty sure I saw this exact picture on the wall in the movie Psycho.
My favorite post of the year
"As well as stupid, Rattati is incredibly slow and accident-prone, and cannot even swim"
|
|
|
CCP Rattati
C C P C C P Alliance
11477
|
Posted - 2014.11.17 08:32:00 -
[9] - Quote
Thanks for the feedback everyone
"As well as stupid, Rattati is incredibly slow and accident-prone, and cannot even swim"
|
|
|
|
|